How Informational Measures Inform Your Practice
(+ Are The Basis of Change)
Douglas A Colquhoun, MB ChB, MSc, MPH
Clinical Lecturer in Anesthesiology
Division of Critical Care Medicine
Department of Anesthesiology
University of Michigan Medical School
Aim for today:
• Variation in our processes of care exists
– Some is the difference between best practice and not.
– Much of it reflect diversity of practice in the absence
defined best practice.
• We need to measure variation
– Variation of Process contributes to Variation of
Outcome
• We need to empower your discussions within
your practice
Patient Factors
Provider Factors
Institutional Factors
Unmeasured Factors
Variation in Process
Variation in Outcome
Types of Measures: Current
• PROCESS Measures:
– PULM 01 - Percentage of cases with median tidal volumes less than 10 ml/kg
– BP 01 - Percentage of unique case-staff instances that sustained intraoperative hypotension was
avoided
• OUTCOME Measures:
– AKI 01 - Percentage of cases that creatinine does not increase more than 1.5 times within 7 days.
– MED 01 - Percentage of cases that avoided did not require the use of nalaxone or flumazenil
• Both have a standard and are either MET or NOT MET.
• We assign thresholds for success of the measure (usually 90%)
• Purpose: Ensure Certain Practices Are Met (Process Measures)
Ensure Certain Outcomes Are Achieved (Outcome Measures)
• Drawback: But don’t comment on WHAT variation exists (or WHY)
Understand our Current Measures
• Many of measures are PROCESS measures, some are
OUTCOME
• PROCESS Measures are typically uncontroversial practices
• Our threshold is typically 90%
• 10% can be outlier cases:
– Errors in documentation
– “Edge cases” not factored into measure design
– Exceptions to the “rule”
– If cases fail… need to ask WHY? Is there a SYSTEMIC cause?
• All of these hypothetical providers MET THRESHOLD
– We don’t make differentiations between them
– But where is the opportunity to change our practice?
Cases
MET
NOT MET
Current Context
• Many of our measures are AT or VERY
NEAR threshold at BCBS sites
• Differences of 1-2% are not significant
until VERY HIGH sample sizes
encountered
• DECISION: Where do focus our energy?
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
0 100 200 300 400 500 600 700 800 900
Probability
Number of Samples
95% Confidence Interval Around 90% Threshold
Types of Measures: Additional Tool
• INFORMATIONAL Measures:
– Opioid Equivalence
– PEEP
• Don’t have a STANDARDS or THRESHOLDS
• Purpose: Allow comparisons about approaches
to aspects of case management
Facilitate discussion within and across
sites
Picture Credit:
bar chart by Musmellow from the Noun Project
Hospital by Raden Situbondo from the Noun Project
doctors by Wilson Joseph from the Noun Project
Comparison
Measure Type
Process
Outcome
Informational
Idea
Did
we do what we
thought
?
Did what we wanted
happen?
How much variation
was
there?
Outcome
MET vs
NOT MET
MET vs
NOT MET
Continuous Values
Threshold
Y
Y
N
Evidence Base
Strong
Strong
Emerging
Response
Look for systemic issues for
significant LOW performance
W
hy this is our practice?
What is best practice?
EXAMPLE: Opioid Equivalency
• Harms of opioid use are well known
• Opioids are essential to peri-operative pain management
• No line exist between MEETING or NOT MEETING a standard
regarding peri-operative opioid use
• Likely not possible to build a robust PROCESS or OUTCOME
measure in this domain
• There are many ways of performing anesthetic care, some of
which lean to greater or lesser extent on opioids
– Sometimes there are compelling reasons for each technique
drugs by Arthur Shlain from the Noun Project
EXAMPLE: Opioid Equivalency (2)
• Compare Similar Cases: Limited by CPT Code “Buckets”
Not Exhaustive List of Cases
Focus on “High Volume” Cases
• Adjusting for Case Length: Reported per hour
• Adjusting for Patient Factors: Reported per kg
Opioid Equivalency (3): Within Institutions
0.62 per kg per hr
0.95 per kg per hr
Difference between orange arrows is x2
Opioid Equivalency (4): Between Institution
Summary
Clinical
Care
Measurement
of Variation
Informed
Discussion
Practice Change
Thank you
